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All  limitations  commonly  associated  with  imaging,  such  as  resolution,  field  of  view,  and  depth  of  field, 
arise  from  linear  theoryfl].  Nonlinear  optics  can  break  these  limits  by  exploiting  the  presence  and 
interaction  of  many  photons  at  once.  To  date,  nearly  all  nonlinear  imaging  techniques  have  relied  on  point 
processes,  such  as  two-photon  fluorescence[2]  or  harmonic  effects[3],  in  which  the  temporal  frequency  is 
the  relevant  parameter.  These  methods  ignore  the  spatial  content  of  the  object,  typically  require  scanning 
to  record  a  whole  image,  and  remain  restricted  by  linear  propagation  from  the  sample  to  the  detector. 
Spatial  nonlinearity  can  overcome  these  issues  by  mixing  modes  with  high  and  low  spatial  frequencies. 

In  this  proposal,  we  explored  the  dynamics  of  imaging  using  spatial  nonlinearity.  This  included 
fundamental  dynamics  of  spatial  mode  mixing,  a  nonlinear  generalization  of  Abbe’s  1873  theory  of 
diffraction,  phase  retrieval  using  nonlinear  diversity,  and  nonlinear  signal-noise  coupling.  Below,  we 
highlight  each  of  these  areas. 


Nonlinear  Abbe  Theory[4] 

The  limitations  of  linear  imaging  were  formalized  succinctly 
by  Ernst  Abbe  in  1873  [1],  In  his  theory,  an  object  is  treated 
as  an  ensemble  of  Fourier  modes,  each  of  which  acts 
individually  as  a  diffraction  grating.  The  observation  of 
spatial  features  is  then  determined  by  the  wavelength  of  the 
illumination  light,  which  governs  diffraction  from  the 
(virtual)  grating,  and  the  acceptance  angle  of  system,  given 
by  the  numerical  aperture.  Spatial  modes  can  be  detected 
only  if  the  wavenumbers  of  the  lowest  diffraction  orders  lie 
within  the  spatial  bandwidth  of  the  system  (Fig.  la). 

Otherwise,  they  —  and  the  corresponding  features  —  are 
lost. 

Interestingly,  Abbe’s  theory  also  suggests  means  of 
imaging  beyond  the  diffraction  limit.  Perhaps  the  most 
straightforward  is  the  use  of  an  additional  diffraction 
grating[5]  to  shift  high  wavenumbers  back  into  the  field  of 
view.  Numerical  processing  can  then  reverse  the  shift  to 
enable  super-resolution  of  the  object.  Other  computational 
methods  also  rely  on  a  priori  information,  including  “extra” 
knowledge  of  the  field[6,  7],  aperture  location[8],  transfer 
function[9,  10],  and/or  the  illuminationfl  1]  or  source[12-14] 
radiance.  Nonlinear  sources  and  objects  have  been  used[2, 

3],  but  to  date  the  methods  have  considered  only  temporal 
frequency  mixing.  These  are  point  processes  that  circumvent  linear  limits  by  generating  shorter 
wavelengths,  tighter  focal  spots,  and  less  unwanted  scattering.  However,  beam  propagation  from  the 
sample  to  the  detector  is  still  linear,  so  that  observations  are  still  restricted  by  the  numerical  aperture  of 
the  system. 

Abbe  theory  can  be  generalized  in  a  straightforward  manner  to  include  spatial  nonlinearity.  The 
mechanism  is  best  understood  as  a  nonlinear  version  of  structured  illumination  (SI).  In  SI,  a  pattern  of 
light  (typically  periodic)  is  projected  onto  the  object,  and  the  resulting  Moire  fringes  let  high -k  modes 
originally  outside  of  the  observation  window  scatter  into  it.  Numerical  deconvolution  is  then  used  to 


Figure  1.  Linear  vs.  nonlinear  Abbe  theory,  (a)  In 

conventional,  linear  Abbe  theory,  object  modes  are 
treated  as  diffraction  gratings.  High-&  modes  are 
blocked  by  the  system’s  numerical  aperture,  so  that  the 
grating  is  not  seen  in  the  image  plane,  (b)  In  the 
presence  of  spatial  nonlinearity,  wave  mixing  generates 
daughter  modes  (striped),  some  of  which  can  propagate 
to  the  detector  plane.  Interference  is  recorded  and, 
with  suitable  knowledge  of  the  medium  response, 
computation  can  recover  the  original  spatial  features. 
O:  object;  Li,L2:  lenses;  SF:  spatial  filter;  MLin,  Mnl: 
Linear  and  nonlinear  measurements,  D:  detector. 


reconstruct  the  original  signal  modes[15].  However,  removal  of  phase  ambiguity  requires  lateral  shifting 
of  the  grating,  and  2D  coverage  in  k- space  requires  rotation  and  multiple  grating  periods.  In  the  nonlinear 
case  (Fig.  lb),  the  object  itself  creates  a  spatially  varying  intensity  pattern,  7(x),  which  in  the  nonlinear 
medium  induces  an  index  change  An  =  An{I).  This  index  change  is  effectively  a  diffraction  grating,  which 
by  construction  is  inherently  phase-matched  with  the  modes  of  the  object. 

Structured  illumination  can  mimic  the  effects  of  nonlinearity  by  projecting  a  nonlinear  (i.e.  non- 
sinusoidal)  pattern  onto  the  object[16].  In  this  approach,  the  mixing  of  high  and  low  wavenumbers  occurs 
in  a  single  plane.  With  bulk  dynamics,  there  are  three  effects  which  work  in  concert:  1)  generation  of 
daughter  waves  from  mode  coupling,  2)  changes  in  the  parent  waves,  and  3)  continuous  evolution  of  (1) 
and  (2)  due  to  propagation.  This  latter  property  can  both  amplify  the  original  changes  as  well  as  cascade 
them.  While  the  particular  wave  mixing  is  object-dependent,  its  general  features  can  expressed  by  a  single 
nonlinear  propagator[17,  18].  Once  a  given  medium  is  characterized,  numerical  reconstruction  (back- 
propagation)  uses  only  measurements  at  the  output  and  works  for  any  object  at  the  input[17,  19]. 

As  an  experimental  proof  of  principle,  we  use  as  an  input  a  grating-probe  signal  whose  grating 
frequency  is  too  high  to  pass  through  the  filter  (Fig.  2).  Here,  the  probe  to  grating  intensity  ratio  is  10:1. 
In  the  linear  case,  the  grating  modes  are  blocked,  and  only  the  uniform  intensity  of  the  plane-wave  probe 
beam  is  measured.  In  the  nonlinear  case,  the  grating  modes  remain  blocked,  but  a  new  mode,  kNL,  is 
generated  and  detected.  Interestingly,  in  this  simple  example,  numerical  back-propagation  is  not 
necessary  to  reconstruct  the  original  modes.  Because  the  system  is  known  to  be  only  weakly  nonlinear, 
the  dominant  process  is  first-generation  four- wave  mixing.  For  a  Kerr  nonlinearity,  the  grating  mode  kg 
will  couple  with  the  probe  kp  to  generate  a  mode  at  kNL  =  2kp~kg,  which  can  be  identified  directly.  By 
extension,  this  argument  gives  an  upper  bound  to  the  resolution  improvement  in  the  weakly  nonlinear 
case.  Only  modes  that  generate  kNL  within  the  observable  spectral  window,  below  the  filter  cutoff  kcut,  can 
be  measured,  i.e.  modes  kNL  <  kcut  corresponding  to  kg  < 
kcut  +  2kP.  Since  the  probe  beam  kP  also  lies  in  the 
observable  window,  its  maximum  value  is  /ccut  as  well. 

Therefore,  the  upper  frequency  bound  of  a  grating  mode  is 
3kcut,  giving  a  resolution  that  is  3x  that  achievable  in  the 
linear  case. 

The  improvement  in  resolution  due  to  nonlinearity  is 
theoretically  unlimited[16],  though  some  subtleties  must 
be  considered.  On  one  hand,  material  response  is  separate 
from  diffraction,  so  that  nonlinear  wave  mixing  works  well 
beyond  any  paraxial  approximation [20].  (Even  adding 
high  spatial  harmonics  numerically  can  be  useful  for 
imaging[21].)  Physically,  evanescent  waves  at  interfaces 
can  be  controlled  nonlinearly[22,  23],  and  structured 
illumination  can  be  enhanced  by  saturation[16]  and  other 
nonlinear  effects.  On  the  other  hand,  the  nonlinear  modes 
generated  by  the  response  are  still  affected  by  diffraction; 
modes  at  the  edges  of  the  transmission  window,  in 
particular,  compete  with  that  part  of  the  system  transfer 
function.  For  a  single  image,  the  end  result  is  a  resonance 
effect  (common  to  all  nonlinear  problems)  in  which  the 
optimum  represents  a  balance  between  the  modes  of  the 
object,  the  information  capacity  of  the  system,  and  the 
nature  of  the  medium  response  [24].  As  in  structured 
illumination,  though,  processing  multiple  images  using 
successively  wider  windows  of  spatial  frequency  can 
compound  the  improvements.  A  detailed  discussion  of 
these  issues  can  be  found  in  the  Supplement  of  Ref.  [4], 


Experiment 


Figure  2:  Bandwidth  extrapolation  via  spatial 
nonlinearity.  Left  column,  simulation;  right  column, 
experiment.  a,b,c);  j,k,l)  Input  signal.  (a,j)  input  grating, 
(b,k)  input  grating  +  plane-wave  probe  beam,  (c,l)  A>space 
measurement,  showing  three  main  wavenumbers.  d,e); 
m,n)  Filtered  linear  signal  (spatial  filter  SF  closed), 
showing  lost  grating  frequencies.  f,g)  ;o,p)  Filtered 
nonlinear  output:  grating  modes  are  still  lost,  but  a 
nonlinearly  generated  daughter  wave  appears.  Crosses 
mark  calculated  reconstruction  of  lost  frequencies  using 
only  the  two  measured  modes.  h,i);  q,r)  Full  object  (SF 
open).  Measured  grating  mode  locations  match 
calculation  from  nonlinear  measurement.  Scale  bar:  85 
pm  for  all  real  space  images  and  n/4.5  pm-1  for  all  k-space 
images  except  i,r),  which  are  demagnified  1.5  times. 


Output  image  Output  spectrum  Filtered,  normally  incident  Filtered,  angled 


Figure  3:  Super-resolution  via  spatial  nonlinearity,  a)  Measured  input  and  b,c)  measured  output  intensities  for  (b)  linear  and  (c)  nonlinear 
propagation.  d,e)  Measured  output  spectrum  for  (d)  linear  and  (e)  nonlinear  propagation.  f,g)  Linear  and  nonlinear  intensity  outputs  with  the 
spatial  filter  SF  closed  and  the  signal  normally  incident  on  the  crystal.  The  SF  allows  the  green  circled  regions  in  d,e)  to  pass.  h,i)  Linear  and 
nonlinear  intensity  outputs  with  the  spatial  filter  SF  closed  and  the  signal  propagating  at  an  angle.  The  SF  allows  the  blue  circled  regions  in  d,e) 
to  pass.  Scale  bar:  200  pm. 


In  the  grating  example,  it  was  known  ab  initio  that  the  number  of  high-7  modes  outside  the  numerical 
aperture  was  limited.  That  is,  there  were  no  modes  7>  and  k»  such  that  the  beat  frequency  k»—  k> 
appeared  inside  the  observation  window.  For  a  more  general  object,  this  cannot  be  guaranteed.  Even 
worse,  there  may  be  many  such  modes  (potentially  an  infinite  number,  though  their  coupling  is 
considerably  more  inefficient  as  the  wavenumbers  increase),  as  well  as  higher-order  interactions.  The 
former  problem  may  be  surmounted  by  changing  the  angle  of  the  probe  beam  7P;  modes  that  mix  with  7P 
will  shift  along  with  it,  while  independent  beat  modes  will  not.  In  this  regard,  we  note  that  a  probe  beam 
should  be  added  to  any  object,  both  to  rule  out  independent  beating  and  to  facilitate  mode  coupling  in  the 
first  place[18].  For  the  problem  of  higher-order  interactions,  the  intensity  of  the  object  or  illumination  can 
be  varied,  as  the  intensity  of  the  output  depends  on  the  product  of  input  amplitudes.  However,  recent 
experiments  have  shown  that  four-wave  mixing  is  the  dominant  interaction  during  propagation[25],  so 
that  neglecting  higher-order  interactions  is  usually  a  valid  approximation. 

Besides  resolution,  there  are  other  metrics  of  image  quality  that  may  take  precedence,  e.g.  visibility  of 
features  for  (automated)  discrimination.  An  example  is  shown  in  Fig.  3.  In  the  linear  case,  the  output 
image  is  again  diffraction-limited  by  the  spatial  filter,  making  the  bars  of  the  chart  unrecognizable.  In  the 
nonlinear  case,  the  bars  are  clear  and  distinct.  The  improvement  can  be  quantified  by  the  Rayleigh 
criterion,  which  corresponds  to  a  minimum  visibility  V  =  (7max  -  7mi„)/  (7max  +  7min)  =0.15  for  detection  of 
the  bars  (from  maxima  to  central  dip).  For  our  experimental  measurements,  the  best  linear  visibility  is 
only  0.095,  while  the  worst  visibility  in  the  nonlinear  regime  is  0.32.  This  is  more  than  a  threefold 
improvement,  a  result  due  not  only  to  high-7  modes  folding  into  the  observation  window  but  also  to  low-7 
modes  (esp.  the  7=0  DC  term)  scattering  out  of  it.  Spectral  energy  coupling  can  be  observed  directly  by 
adjusting  the  spatial  filter.  When  the  filter  is  centered  on  the  optic  axis  (7  =  0),  the  nonlinear  output  power 
is  significantly  lower  than  the  linear  power.  When  the  filter  is  shifted  laterally  by  5mrad,  so  that  it  is 
centered  on  the  first-order  diffraction  lobe,  the  contrast  and  power  are  greatly  increased.  Interestingly,  this 
improvement  is  object-dependent,  implying  that  system  performance  can  be  optimized  if  the  target  is 
known.  More  generally,  the  results  show  that  fundamental  trade-offs  in  linear  optics,  such  as  resolution 
vs.  contrast,  need  not  apply  in  nonlinear  imaging  systems. 


Nonlinear  phase  retrieval[26] 


One  of  the  earliest  and  most  important  examples  of  computational 
imaging  is  phase  retrieval.  The  most  common  example  is  the  Gerchberg- 
Saxton  algorithm[27],  in  which  the  phase  is  computed  using  intensity 
patterns  measured  at  two  different  planes  of  propagation  [typically  the 
near-field  (image)  and  far-field  (Fourier)  planes].  Other  forms  of 
parameter  diversity,  such  as  axial[28]  and  transverse[29,  30] 
displacement,  wavelength[31],  and  polarization  [7],  have  been  used  as 
well.  These  methods  work  because  in  beam  propagation,  dispersion  and 
diffraction  of  spatial  modes  convert  phase  information  into  intensity. 

Nonlinear  media  provide  an  additional  relationship  between 
amplitude  and  phase  through  intensity-dependent  changes  to  the 
refractive  index.  As  a  result,  nonlinear  systems  provide  a  natural  route 
toward  contrast  enhancement  and  increased  sensitivity  to  phase[26,  32- 
38],  A  comparison  between  linear  and  nonlinear  phase  retrieval  is 
shown  in  Fig.  4.  In  each  case,  a  uniform  phase  (p  =  0  is  used  as  input 
then  iteratively  approximated,  using  the  two  measured  amplitudes  as  boundary  conditions.  Compared 
with  linear  phase  retrieval,  which  is  independent  of  amplitude  and  has  a  monotonic  convergence  behavior, 
nonlinear  retrieval  is  much  more  complex.  Object-  and  intensity-dependent  phase-matching  conditions 
give  rise  to  resonant  spikes  in  convergence.  Typically,  the  largest  spike  sets  a  condition  for  stopping  the 
iteration  cycle,  but  more  work  needs  to  be  done  to  prove  this.  Nevertheless,  in  all  cases  tested  so  far, 
nonlinearity  yielded  a  faster  convergence  speed  and  a  lower  phase  error  than  linear  methods  (both  ~2x 
better). 


Figure  4.  Nonlinear  phase  matching. 

Phase  matching  gives  a  convergence 
criterion  for  nonlinear  phase  retrieval, 
yielding  faster  convergence  with  lower 
error.  Left:  nonlinear;  right:  linear. 


Nonlinear  signal-noise  coupling[39,  40] 


imaging  system 


Nonlinearity  has  a  similarly  dramatic  effect  on  light  that  is 
partially  spatially  correlated.  As  with  diffraction,  spatial 
coherence  can  either  compete  or  cooperate  with  nonlinearity, 
e.g.  by  inhibiting  instabilities  or  generating  new  ones.  In 
terms  of  information  processing,  there  is  a  rich  interplay 
between  spatial  nonlineairty,  signal,  and  noise  that  is  only 
just  beginning  to  be  explored[24].  For  example,  energy  can 
transfer  nonlinearly  from  noise  to  signal,  resulting  in  a 
“stochastic  resonance”  that  can  boost  the  signal  to 
detectability[41].  In  contrast,  mode  mixing  from  nonlinearity 
can  create  an  effective  noise,  reducing  the  information 
capacity  of  an  optical  system[24,  42], 

An  example  of  nonlinear  signal-noise  coupling  is  shown 
in  Fig.  5.  Light  from  a  532nm  laser  is  incident  on  a  resolution 
chart  followed  by  a  holographic  diffuser.  A  lens  then  images 
the  resolution  chart  onto  a  photorefractive  SBN:60 
(Sro.6Ba0.4Nb206)  crystal.  The  role  of  the  diffuser  is  to  scatter 
light  from  the  object,  in  a  maner  similar  to  clouds  or  tissue, 
so  that  the  chart  features  are  diffused  and  unrecognizable.  In 
the  experiment,  the  diffuser  has  a  Gaussian  angular  spread  of 
0.5  and  is  placed  15mm  after  the  object,  so  that  the 
correlation  length  ~  100pm  at  the  input  face  of  the  crystal. 

To  give  a  uniform,  rather  than  speckled,  input  pattern,  the  diffuser  is  rotated  at  a  rate  ( 
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imaging  system 


Figure  5.  Instability-driven  recovery  of  diffused 
images,  (a)  Experimental  setup,  (b-g)  Experimental 
output  with  increasing  nonlinearity,  (b)  Undiffused 
image  of  the  resolution  chart,  (c)  Output  after  linear 
propagation.  (d)  Nonlinear  output  for  weak 
nonlinearity  ( Sn  =  1.3x10“^).  (e-g)  Output  for  stronger 
nonlinearity,  above  the  threshold  for  incoherent 
modulation  instability  ((e)  Sn  =  1.7xl0“4,  (f)  Sn  = 
2.0x10-4,  (g)  Sn  =  2.3x10^).  Numbers  below  the 
frames  are  the  corresponding  input-output  correlation 
coefficients,  q  _  ^  .  jnlS^  . 


-200Hz)  that  is 


much  faster  than  the  response  time  of  the  crystal  (~ls).  Figure  5b  shows  the  input  pattern  without  the 
diffuser,  while  Fig.  5c  shows  the  time-averaged  pattern  when  the  diffuser  is  rotated.  For  SBN,  the 
nonlinear  index  change  is  An  =  icEavv (7)/(l  +  (/)),  where  (7)  is  an  intensity  perturbation  above  a 
spatially  homogeneous  background  intensity  (70),  £app  is  an  electric  field  applied  across  the  crystalline  c- 
axis,  and  k  =  n0r33(l  +  (70))  is  a  constant  depending  on  the  base  index  of  refraction  n0,  the  electro-optic 
coefficient  r33,  and  (70)  [43],  Here,  the  brackets  (  )  denote  a  time  average.,  where  the  integration  time  x 
is  longer  than  the  time  scale  of  fast  phase  fluctuations  x9  but  shorter  than  the  slow  response  time  xr  the 
photorefractive  crystal  (/. e.  tv  <  x  <  xr).  For  the  experiments,  the  illumination  intensity  is  kept  a  constant 
lOpW  and  a  self- focusing  nonlinearity  is  created  and  controlled  by  varying  an  applied  voltage  bias  across 
the  crystal.  Light  exiting  the  nonlinear  crystal  is  then  imaged  onto  a  CCD  camera. 

Scattered  images  like  the  one  in  Fig.  5c  are  partially  coherent  and  can  be  characterized  as  a  statistical 
ensemble  of  spatial  modes.  For  the  geometry  here,  the  field  passing  through  the  chart  is  effectively 
multiplied  by  many  realizations  of  the  diffuser  transmission  function  (a  random  pure-phase  plate),  so  that 
the  initial  image  cannot  be  reconstructed  by  simple  linear  averaging[44].  On  the  other  hand,  the  modes  of 
the  product  beam  are  not  entirely  random;  there  are  correlations  due  to  the  original  image.  In  a  nonlinear 
medium,  these  modes  can  interact  dynamically  as  they  propagate.  For  simple  (e.g.  homogeneous)  beams, 
possible  results  include  suppressed  or  enhanced  diffraction[45],  instabilities[46-48],  and  spatial  optical 
turbulence[49].  For  beams  with  initial  correlations,  the  dynamics  are  preferentially  biased.  In  particular, 
signal  modes  can  reinforce  each  other  and  extract  energy  from  the  diffuse  background,  growing  as  they 
propagate.  For  example,  a  Kerr-type  nonlinearity  An  =  y  (ip(x)tp*  (x)^ ,  where  y  is  the  nonlinear  coefficient 

and  y/(x)  is  the  wave-field,  gives  a  convolution  of  modes  in  Fourier  space,  An  =  yj  elk1  \k  -  k')\ , 

so  that  any  correlations  in  the  wavefunction  yj  will  influence  the  subsequent  nonlinear  response.  If  an 
instability  is  seeded,  then  exponential  growth  of  the  signal  is  possible. 

An  experimental  demonstration  of  this  dynamics  is  given  in  Fig.  5,  which  shows  the  output  face  of 
the  crystal  as  the  nonlinear  coupling  strength  is  increased.  As  in  the  simulations,  there  is  no  visible 
change  in  the  output  pattern  until  a  threshold  is  reached  (Figs.  5c, d),  after  which  the  quality  of  the  image 
degrades.  A  measure  of  the  image  quality  can  be  obtained  by  computing  the  cross-  correlation 
C  =  J  Uchar,  •  I m  between  an  ideal  input  chart  (Ichart  =  1  at  the  bars  and  zero  elsewhere)  and  the 

measured  output  intensity.  These  values  are  shown  beneath  the  frames  in  Fig.  5. 


Phase-space  optics[50] 

Phase-space  representations  simultaneously  store  spatial  and  spatial  frequency  information,  in  a  manner 
analogous  to  the  position-momentum  representation  in  mechanics.  Indeed,  under  the  common  assumption 
that  the  coherence  time  of  the  wave-field  is  much  shorter  than  the  response  time  of  the  camera  (typical  of 
most  non-laser  sources),  a  Hamiltonian  or  eikonal  description  of  wave  evolution  is  suitable[51]. 
Accordingly,  any  (nonlinear)  dynamics  that  can  benefit  from  a  Hamiltonian  description  would  best  be 
treated  by  a  phase-space  representation. 

There  are  many  choices  of  functions  that  represent  phase  space.  Perhaps  the  most  popular  is  the  time- 
averaged  Wigner  distribution  function  (WDF),  defined  as 

f(r, *)  =  /(V* V  +  f  /  2  W  -  §  /  T)e**)d% 

where  r  =  (x,y)  and  k  =  (kx,  ky)  are  the  2D  spatial  and  spatial  frequency  vectors,  respectively[52].  This 
form  is  essentially  a  Fourier  transform  of  the  two-point  correlation  function,  so  retains  local  information 
about  intensity  and  momentum  (Poynting  vector).  In  the  coherent  limit,  the  WDF  is  highly  redundant, 
since  coherent  light  at  a  single  position  is  associated  with  only  a  single  direction  (phase).  Only  a  2D 


Figure  6.  Experimental  creation  and  measurement  of  beams  with  arbitrary  spatial  coherence.  An  input  spatial  light  modulator  (SLM) 
creates  a  time-multiplexed  speckle  beam  and  a  measurement  SLM  uses  a  scanning  aperture  to  record  a  windowed  Fourier  transform  of  the  time- 
averaged  pattern.  Subplots  show  the  local  Fourier  power  spectra  (spatial  frequency  distributions)  as  a  function  of  position.  Top:  local  power 
spectrum  at  different  points  in  (x,y).  Bottom:  intensity  image  of  one  representation  of  a  random  speckle  pattern  at  the  input  (diffuser)  SLM.  For 
comparison,  the  standard  measurements  of  real-space  and  Fourier-space  images,  taken  without  using  the  aperture  on  the  output  SLM,  are  shown 
to  the  right  of  each  figure,  a)  Elliptical  speckles,  which  rotate  across  the  beam,  b)  Gaussian  speckles,  with  varying  widths  across  the  beam. 


description  is  necessary,  with  two  projections  of  the  WDF  being  sufficient  to  recover  the  entire  wave-field 
uniquely[53],  In  contrast,  phase-space  distributions  of  partially  coherent  beams  are  usually  not  redundant,  as 
coherence  information  fills  up  more  of  the  4D  phase  space[54]. 

Until  our  work,  nearly  all  measurements  in  linear  optics,  and  all  those  in  nonlinear  optics,  recorded 
only  the  marginals  of  the  WDF,  i.e.  the  intensity  and  power  spectrum  projections  I(r)  =  J  f(r,k)dk  and 

I(k)  =  f  f(r,k)dr 

This  means  that  coherence  properties  that  change  as  a  function  of  transverse  position,  as  well  as  more 
complex  phase-space  structures,  have  eluded  observation.  Correspondingly,  there  has  been  limited 
motivation  to  generate  arbitrary,  spatially  varying  patterns  of  coherence,  especially  for  two-dimensional 
beams.  We  remedied  both  problems  in  this  proposal,  using  spatial  light  modulators  for  both  coherence 
synthesis  and  phase-space  measurement. 

Examples  of  4D  coherence  control  and  measurement  are  shown  in  Fig.  6.  Figure  6  shows  two 
different  beams  that  have  homogeneous  real  space  intensities  but  varying  coherence  properties  across 
both  axes  of  r.  The  first  example  (Fig.  6a)  uses  elliptically  shaped  speckles,  whose  orientation  is  rotated 
in  different  parts  of  the  field  of  view.  For  clarity,  we  show  only  one  realization  of  an  ensemble  of  50 
speckle  patterns  having  the  same  spatial  statistics  but  different  random  implementations  of  speckle;  the 
integrated  real-space  intensity  contains  no  information  about  the  coherence  of  the  beam,  while  the 
Fourier-domain  intensity  is  a  projection  over  all  r  and  cannot  reveal  the  spatial  variations  of  coherence 
properties.  In  contrast,  local  /(-space  images  (taken  by  the  Fourier  CCD  with  the  SLM  aperture  in  different 
locations)  do  give  information  about  the  local  coherence  properties.  A  similar  example  is  given  in  Fig  6b, 
which  shows  a  beam  designed  to  have  anisotropic  Gaussian  statistics  with  an  ellipticity  that  varies 
according  to  position.  Two-dimensional  beams  with  such  local  variations  of  coherence  properties  had  not, 
to  our  knowledge,  been  created  and  measured  previously. 


Conclusions 

In  this  proposal,  we  used  spatial  nonlinearity  as  a  degree  of  freedom  for  imaging.  The  mode  mixing  which 
results  enables  the  breaking  of  many  fundamental  limits  of  conventional  (linear)  imaging,  including 
resolution  and  field  of  view,  and  can  overcome  many  of  the  basic  trade-offs,  including  resolution  vs. 
contrast  and  signal  vs.  noise.  In  terms  of  optical  science,  the  work  expanded  Abbe’s  1873  theory  of 


diffraction  to  include  spatial  nonlinearity,  prompted  the  development  of  new  experimental  methods  of 
phase-space  measurement,  and  facilitated  the  discovery  of  new  nonlinear  dynamics.  In  terms  of  imaging, 
the  results  generalized  the  field  of  computational  imaging,  on  both  the  device  and  algorithmic  levels. 
They  also  introduced  many  new  and  outstanding  issues  that  need  to  be  addressed,  such  as  the  best  types 
of  nonlinearity  to  use,  the  uniqueness  and  robustness  of  solutions,  the  levels  of  improvement  possible,  and 
the  information  capacity  of  nonlinear  systems.  These  will  be  explored  in  future  work. 


Appendix:  Publications  resulting  from  the  grant 


1.  “Nonlinear  Abbe  theory” 

Christopher  Barsi  and  Jason  W.  Fleischer,  Nature  Photonics  7,  639  (2013). 

2.  “Quantitative  amplification  of  weak  images  by  nonlinear  propagation” 

Laura  Waller,  Dmitry  V.  Dylov,  and  Jason  W.  Fleischer,  Optics  Letters  38,  82  (2013). 

3.  “Phase  retrieval  using  nonlinear  diversity” 

Chien-Hung  Lu,  Christopher  Barsi,  Matthew  O.  Williams,  Nathan  Kutz,  and  Jason  W.  Fleischer, 
Applied  Optics  52,  D92  (2013). 

4.  “Enhancing  layered  3D  imaging  with  a  lens” 

Stefan  Muenzel  and  Jason  W.  Fleischer,  Applied  Optics  52,  D97  (2013). 

5.  “Observation  of  the  kinetic  condensation  of  classical  waves” 

Can  Sun,  Shu  Jia,  Christopher  Barsi,  Sergio  Rica,  Antonio  Picozzi,  and  Jason  W.  Fleischer,  Nature 
Physics  8,  470  (2012). 

6.  “Phase-space  measurement  and  coherence  synthesis  of  optical  beams” 

Laura  Waller,  Guohai  Situ,  and  Jason  W.  Fleischer,  Nature  Photonics  6,  474  (2012). 

7.  “Wrinkles  and  deep  folds  as  photonic  structures  in  photovoltaics” 

Jong  Bok  Kim,  Pilnam  Kim,  Nicolas  C.  Pegard,  Soong  Ju  Oh,  Cherie  R.  Kagan,  Jason  W.  Fleischer, 
Howard  A.  Stone,  and  Yueh-Lin  Loo,  Nature  Photonics  6,  327  (2012). 

8.  “Spectral  dynamics  of  spatially  incoherent  modulation  instability” 

Can  Sun,  Laura  Waller,  Dmitry  V.  Dylov,  and  Jason  W.  Fleischer,  Physical  Review  Letters  108, 
263902  (2012). 

9.  “Nonlinear  restoration  of  diffused  images  via  seeded  instability”  (Invited) 

Dmitry  V.  Dylov,  Laura  Waller,  and  Jason  W.  Fleischer,  IEEE  Journal  of  Quantum  Electronics  18, 
916(2012). 

10.  “Instability-driven  recovery  of  diffused  images” 

Dmitry  V.  Dylov,  Laura  Waller,  and  Jason  W.  Fleischer,  Optics  Letters  36,  3711  (201 1). 

11.  “Optimizing  holographic  data  storage  using  fractional  Fourier  transforms” 

Nicolas  C.  Pegard  and  Jason  W.  Fleischer,  Optics  Letters  36,  2551  (2011). 

12.  “Diffraction  from  an  edge  in  a  self-focusing  medium” 

Wenjie  Wan,  Dmitry  V.  Dylov,  Chris  Barsi,  and  Jason  W.  Fleischer,  Optics  Letters  35,  2819  (2010). 
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